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Abstract 

Let K/F be a quadratic extension of p-adic fields, and n a positive integer. A smooth 
irreducible representation of the group GL{n, K) is said to be distinguished, if it admits on its 
space a nonzero GL{n, F)-invariant linear form. In the present work, we classify distinguished 
generic representations of the group GL{n, K) in terms of inducing quasi-square-integrable 
representations. This has as a consequence the truth of the expected equality between the 
Rankin-Selberg type Asai L-function of a generic representation, and the Asai L-function of 
its Langlands parameter. 

Introduction 

Given K/F a quadratic extension of p-adic fields, we denote by a the non trivial element of the 
Galois group of K over F. We denote by r/j^/p the character of order 2 of F*, trivial on the set 
of norms oi K* . 

A smooth representation of GL{n, K) is said to be distinguished if it admits on its space a 
nonzero linear form, which is invariant under GL{n, F). The pair {GL{n, K), GL{n, F)) is known 
to be a generalized Gelfand pair, which means that for an irreducible representation (tt, K-) of 
GL{n, K), the space of GL(n, i^)-invariant linear forms on is of dimension at most one. The 
unitary distinguished representations are the natural space which supports the Plancherel measure 
of the symmetric space GL{n, F)\GL{n, K). Hence their understanding is related to harmonic 
analysis on GL{n, F)\GL{n, K). 

We classify here distinguished generic representations of GL{n,K), in terms of inducing dis- 
crete series representations. More precisely we prove the following result. 

Theorem 14. 2L Let n be a generic representation of the group GL{n,K), obtained by normalized 
parabolic induction of quasi-square-integrable representations Ai,...,A(. It is distinguished if 
and only if there exists a reordering of the Ai 's, and an integer r between 1 and t/2, such that 
we have Af_^]^ = for i = 1, 3, .., 2r — 1, and Ai is distinguished for i > 2 



r. 



Distinguished representations of GL{n, K) are also related by a conjecture of Flicker and Ral- 
lis, to the base change theory of representations of a unitary group (see for example |A-Rj l. The 
main result of this paper could perhaps reduce the proof of this conjecture for generic distin- 
guished representations of GL{n, K), to the case of discrete series representations. 

Generic distinguished representations are related to the analytic behaviour of meromorphic 
functions, called Asai L-functions associated with generic representations. 

The basic theory of the Asai L-function of a generic representation tt of GL{n,K), denoted by 
Lp{tt) and defined as the gcd of functions obtained as meromorphic extension of Rankin-Selberg 
integrals, such as its functional equation, has first been developed by Flicker in jFl) and |F3| . 



Then in [K], Kable proves that if the Asai L-function Lp (tt) of a discrete series representation 
TT admits a pole at zero, then the representation tt is distinguished. This, with the equality of 
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the product of the Asai L-functions of tt and rjn (for a character 77 of K* extending tjk/f), and 
of the classical L-function of the pair (7r°',7r^), obtained by a global- local method, allows him to 
prove the so called Jacquet conjecture for discrete series representations. This result states that 
a discrete series representation tt of GL{n,K) which is Galois autodual (i.e. tt^ = tt'"), is either 
distinguished or ?7/f/j?-distinguished. 

This result is specified in |A-K-Tj . where it is shown that the preceding (either/or) is exclusive, 
by first proving that the Asai L-function of a tempered distinguished representation has a pole 
at zero. 

For non discrete series representation, the correct statement is that of Theorem 14.21 Actually 
this theorem can be seen as a generahzation of Jacquet's conjecture for discrete series represen- 
tations, as it says that a generic representation of GL{n, K) is Galois autodual if and only if 
it is parabolically induced from three representations, one that is distinguished but not rix/p- 
distinguished, one that is distinguished and 77^/i?-distinguished, and one that is distinguished but 
not ry^f /^-distinguished. Among these, the distinguished are those with purely ?7x/_F-clistinguished 
part equal to zero. 

The last step before Theorem 14.21 consisting of showing that the representations described in the 
theorem are indeed distinguished is the main result of jM4j . 

Concerning Rankin-Selberg type Asai L-functions, a definitive statement relating their poles 
and distinction is obtained in [M2], where it is proved that a representation tt is distinguished, if 
and only if its Asai L-function admits a so called (in the terminology of |C-P) ) exceptional pole 
at zero. 

There are two other ways to associate an Asai L-function to a representation tt of the group 

GL{n,K). 

The first is by considering the n-dimensional representation p of the Weil-Deligne group W'j^ of 
K, associated to tt by the local Langlands correspondence. One then defines by multiplicative 
induction a representation of the Weil-Deligne group W'p (which contains W'p^ as a subgroup of 

index 2) , of dimension , denoted by M^^^f (p) . The Asai L-function corresponding to p is by 
definition the classical L-function of the representation M^f (p), which we denote by Lp{p). 

The second, called the Langlands-Shahidi method, is introduced in [Sh]. We denote by Lp'^ (tt) 
the meromorphic function obtained by this process, the study of its poles is this time related to 
the fact of knowing when a representation tt is obtained by base change lift from a unitary group 
(see [Go]). It is conjectured that these three functions are actually the same (cf. [Hej . [K], |A-Rj l. 
Henniart proved in jHe j that the functions L^,' (tt) and Lp{p) are equal for every irreducible 
representation tt. Anandavardhanan and Raj an proved in |A-R) that the functions L^(7r) and 
Lp'^ (tt) are equal for tt in the discrete series of GL{n,K). 

In |M3j . which can be used as a survey for local Rankin-Selberg type Asai L-functions, The- 
orem 14.21 is stated as a conjecture. It is then showed using a method of Cogdell and Piatetski- 
Shapiro and the known equality of Lpiir) and Lp{p) for discrete series representations, that the 
theorem impHes the equality of Lp{n) and Lp[p) for generic representations. 
Hence we have the following result. 

Theorem l4.3L Let n be a generic representation of the group GL{n, K), and let p he the represen- 
tation of dimension n of the Weil-Deligne group W'^ of K , corresponding to tt through Langlands 
correspondence. The following equality of L-functions is satisfied: 

Lf(^,s)=Lf(p,s). 

Now the main tool for the proof of Theorem 14.21 classical in this situation, is Mackey theory. 
For instance the case of principal series representations is treated in section 6 of |JLR| . 



2 



A generic representation tt of the group GL{n,K), is obtained by normalized parabolic induc- 
tion of a discrete series representation A = Ai(g)---(g)At of a standard Levi subgroup of a 
parabolic subgroup P{K) of GL{n,K). Calling R a set of representatives of the double classes 
P{K)\GL{n, K)/GL{n, F), the restriction to GL{n, F) of the representation tt has a factor series, 
with factors being induced representations of ^\p(K)nuGL{n.F}u-^ to the group uGL{n, F)u'~^ , 
for some u in R. If the representation tt is GL{n, F)-distinguished, then it is at least the case for 
one of the factors. But using Frobenius reciprocity law, such a factor is distinguished, if and only 
if the representation A is itself x-distinguished by P{K) n uGL{n, F)u~^ fore some character x- 
Studying the structure of this subgroup, implies that the A^'s must be of the requested form. The 
group structure of P{K) n uGL{n, F)u~^ was studied in |JLR| when the element uu~'^ (which is 
actually an involution in 6„) normaHzes the standard Levi subgroup of P. Here we reduce the 
general case to this particular case. 

The first part of Section 1 is about definitions and notations of the basic objects we use. 
The second part concerns itself with results of Bernstein and Zelevinsky about classification of 
discrete series representations in terms of segments, and the computation of their Jacquet modules. 
A set of representatives R of the double classes P{K)\GL{n, K)/GL{n, F) for a standard paraboHc 
subgroup P{K) of GL{n,K) was already described in |JLRj . In Section 2, we give a geometric 
way of obtaining this set, which has its advantage. We then show how to reduce the study of 
the group structure of P{K) n uGLin, F)u^^ to the admissible case treated in [JLRj . We end by 
computing the modulus character of this group. 
We eventually prove Theorem 14.21 in the last section. 
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1 Preliminaries 

1.1 Notations and definitions 

We fix until the end of this paper a local field F of characteristic zero. We fix a quadratic exten- 
sion of i^. 

If G is a group acting on two vector spaces V and V , then HomQ{V, V) designates the space 
of G-equivariant morphisms from V ioV . 

If is a finite extension of F, we denote by ve the discrete valuation of E, which verifies that 
ve{t^e) is 1 if tte is a prime element of E. We denote by qe the cardinality of the residual field of 
E. We denote by | \e the absolute value of E defined by \x\e = Qe^'^^^^ ^ ^ ^- denote 
by Re the valuation ring of i?, and by Pe the maximal ideal of Re- Finally we denote by We 
the Weil group of E (cf. [^), and by W'^ the Weil-Deligne group of E. The group W'^ is the 
semi-direct product We x SL{2,C), with We acting by its quotient group Qe on SL{2,C). A 



Let G be an affine algebraic group defined over the field F. If E is an extension of F, we 
denote by G{E) the group of the i?-points of G. Such a group is locally compact and totally 
disconnected, we will call it an Z-group. 

Let n be a positive integer, we denote by Af„ = Mn{F) the additive group ofnxn matrices with 



Frobenius element in We acts on 5*^(2, C) is by conjugation by the matrix 
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entries in F, and we denote by G„ the general linear group GL{n,F) of invertible matrices of 
Mn{F). If M belongs to M„, we denote its determinant by det{M). 

We call partition of a positive integer n, a family n = {ni, . . . ,nt) of positive integers (for a 
certain f in N — {0}), such that the sum ni + • ■ ■ + rit is equal to n. To such a partition, we 
associate an algebraic subgroup of Gn denoted by Pn , given by matrices of the form 

I g\ -k -k -k ★ \ 
52 * * * 

■k -k 1 

9t-i * 
\ 9t J 

with gi in Cm for i between 1 and t. We call it the standard parabolic subgroup associated with 
the partition n. \% call parabolic subgroup any conjugate of a standard parabolic subgroup. We 
denote by A^^ its unipotent radical subgroup, given by the matrices 

/ Im * * \ 

-k 

and by Mn its standard Levi subgroup given by the matrices 

V 9t) 

with the 5,'s in G„^. The group Pn identifies with the semidirect product xi M^. 

For n = (1, . . . , 1), we denote by Bn the Borel subgroup P^, and by T„ its standard Levi 
subgroup. The Lie algebra of Lie{Gn) decomposes as Lie{Tn) © {(BaG'S>Lie{Na)) under the ad- 
joint action of r„. The group T„ acts on each one-dimensional space Lie{Na) by a non-trivial 
character a, and by the trivial character on LieiTn). The set $ is the root system of Gn with 
respect to T„. The group is the unipotent subgroup of G„ with Lie algebra Lie{Na). We 
denote by the roots a such that C i?„ and by $~ the other roots. We denote by $m or 
$fi the set of roots a with 7V„ C M for M = M^, by and the intersections $m n and 
$M n $~ respectively. 

The quotient W = NQ^{Tn) /Tn of the normalizer of T„ in G„ by r„, is called the W^yl group 
of G„, it identifies with the symmetric group ©„, and permutes the roots in $. The quotient 
Wm = NMn{Tn)/Tn of the normalizer of T„ in by T„, is called the Weyl group of M^, it 
identifies with the product group ni=i ^^'^ permutes the roots in $m- 

Let G be an Z-group (i.e. locally compact totally disconnected), we denote by dag or simply dg 
if the context is clear, a left Haar measure G. For x in G, we denote by 5g{x) the positive number 
defined by the relation dg{gx) = 6^^ {x)dg{g). The modulus character Aq defines a morphism 
from G into K>o. We denote by do (which we also call modulus character) the morphism from 
G into M>o defined by a; i— > Ac{x~^). 

Let G be an Z-group, and H a subgroup of G, a representation (tt, V) of G is said to be 
smooth if for any vector v of the vector space V, there is an open subgroup Uy of G stabilizing v 
through TT. We denote by the subspace of fixed points of V under H. The category of smooth 
representations of G is denoted by Alg{G). If (tt, V) is a smooth representation of G, we denote 
by TT^ its dual representation in the smooth dual space V of V. 
We will only consider smooth complex representations of Z-groups. 
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Definition 1.1. Let G be an l-group, H a closed subgroup of G, and (tt, a representation of 
G. If X is a character of H , we say that the representation tt is x- distinguished under H , if it 
admits on its space a nonzero linear form L, verifying L{Tr{h)v) = x{h)L{v) for all vinV and h 
in H. If X = 1; we say H -distinguished instead of 1- distinguished. We omit "H-" if the context 
is clear. 

If p is a complex representation of in V^, we denote by C°°{H\X, p, Vp) the space of functions 
/ from X to Vp, fixed under the action by right translation of some compact open subgroup Uf 
of G, and which verify f{hx) = p{h)f{x) for h e H, and x e X . We denote by G^{H\X, p,Vp) 
the subspace of functions with support compact modulo H of C°°{H\X, p, Vp). 
We denote by Ind^{p) the representation by right translation of G in C°"{H\G, p,Vp) and by 
ind^{p) the representation by right translation of G in Cf[H\G, p, Vp). We denote by Ind' ^{p) 
the normalized induced representation Ind'^{{AG / ^h)^^^ p) and by ind'%{p) the normalized 
induced representation ind^{{/S.G / ^h)^^"^ p)- 

Let n be a positive integer, and n — (rii, . . . , nt) be a partition of rt, and suppose that we have a 
representation {pi,Vi) of G„. [K) for each i between 1 and t. Let p be the extension to Pn of the 
natural representation pi ® ■ ■ ■ ® pt oi Mn — Gni{K) x • • • x Gnt{K), trivial on Nn. We denote 
by pi X • • • X the representation ind''^"^^^ (p) — Ind''^"^^^{p). 

1.2 Segments and quasi-square integrable representations 

From now on we assimilate representations to their isomorphism classes. 

In this subsection, we recall results of Bernstein and Zelevinsky about quasi-square-integrable 
representations, more precisely their classification in terms of segments associated to supercuspi- 
dal representations, and how to compute their Jacquet modules. 

If TT is an irreducible representation of Gn{K), one denotes by its central character. 
We recall that an irreducible representation of Gn{K) is called supercuspidal if all its Jacquet 
modules associated to proper standard Levi subgroups are zero, which is equivalent to the fact 
that it has a coefficient with support compact modulo the center Zn[K) of the group Gn[K). 

An irreducible representation tt of the group Gn{K) is called quasi-square-integrable, if there 
exists a positive character x of the multiplicative group K* , such that one of the coefficients 
g ^ c(g) of TT verifies that c{g)x{det{g)) is a square-integrable function for a Haar measure of 
Gn{K)/Zn{K). One says that the representation tt is square-integrable (or belongs to the discrete 
series of Gn{Kj) if one can choose x to be trivial. 

If p is a supercuspidal representation of Gr{K) for a positive integer r, one denotes by p\ \k the 
representation obtained by twist with the character | det( )\k. 

In general, if E is an extension of F, and x is a character of E*, we will still denote by x the 
character x ° det. 

We call segment a list A of supercuspidal representations of the form 

A = [pi \'k\p\ lt^...,p] 

for a positive integer /. We call length of the segment the integer rl. We have the following 
theorem (Theorem 9.3 of [Z]) that classifies quasi-square integrable representations in terms of 
segments. 

Theorem 1.1. Let p be a supercuspidal representation of Gr{K) for a positive integer r. The 
representation p x p\ |i? x • • • x p| j'^^^ of Gri{K) is reducible, with a unique irreducible quotient 
that we denote by [p\ |^^, • • ■ ,p]- A representation A of the group Gn{K) is quasi-square- 

integrable if and only if there is r G {1, . . . , n} and I G {1, . . . , n} with Ir = n, and p a supercus- 
pidal representation of Gr{K), such that the representation A is equal to [p\ |^^, ■ ■ ■ ,p]- 
The representation p is then unique. 
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A representation of this type is square-integrable if and only if it is unitarizable, or equiva- 
lently if and only if p\ |^ ^■'^^ is unitarizable (i.e. its central character is unitary). We say that 
two segments are linked if none of them is a subsegment of the other, but their union is still a 
segment. 

Now we allow ourselves to call segment a quasi-square-integrable representation, and to denote 
such a representation by its corresponding segment. 

We will also use the following useful notation: if Ai and A2 are two disjoint segments, which are 
linked, and such that Ai precedes A2 (i.e. the segment Ai is of the form [pi \ |^ 
the segment A2 is of the form [P2I Ik \ Pi| Ik- ■ • • 'Ps], with pi = P2I |^), we denote by [Ai, A2] 
the segment [pi\ |'^"\ . . . ,^2]- 

Let P be a standard paraboHc subgroup of Gn{K), AI its standard Levi subgroup, and let P' 
be a standard parabolic subgroup of M, with standard Levi subgroup M', and unipotent radical 
N'. We recall that the normalized Jacquet module of a representation (p, V) of M, associated 
to M', which we denote by rM'.Mip), is the representation of M' on the space V/V{N') (where 
V{N') is the subspace of V generated by vectors of the form v — Tr{n')v for vinV and n' in N'), 
defined by tm' .M{p){m'){v + V{N')) = S^^''^ p{m')v + V{N'). 

The following proposition (Proposition 9.5 of [ZJ), explains how to compute normalized Jacquet 
modules of segments. 

Proposition 1.1. Let p a supercuspidal representation of Gr{K) for a positive integer r. Let A 
be the segment [p\ |^^, p\ |'^^, . . . , p], for a positive integer I. Let M be a standard Levi subgroup 
of Gir{K) associated with a partition (ni, . . . , nt) of Ir. 

The representation rM,G(A) is zero, unless (ni,...,nt) admits (r, . . . , r) as a sub partition, in 

I times 

which case A is of the form [Ai, . . . , At], with A^ of length rii, and rM,G{^) is equal to the tensor 
product Ai (g) • • • (g) At . 

2 Double classes P{K)\Gn{K) /Gn{F) 

Let n be a partition (ni, . . . ,rtf) of a positive integer n, we denote by P the standard paraboHc 
subgroup Pn{K) oi G — Gn{K), by M its standard Levi subgroup, by B the group Bn{K), by 
T the group Tn{K). By abuse of notation, the group Na will be Na{K). We denote by H the 
group Gn{F). 

We study in first place the double classes of H\G/P. This has already been done in Section 6 of 
[JLRj and one can check that the representatives we obtain are the same they obtain (see Remark 
12. ip . This set is described in [JLRj as the involutions in the set of left and right WM-reduced 
elements of W. However we give a more geometric proof in which every representative u of the 
double classes of H\G/P comes naturally equipped with a sub-partition s of (ni,...,nt), or 
equivalently a standard parabolic subgroup Pg of G contained in P. This new standard parabolic 
subgroup will have the nice property that the representative u is Pj-admissible (in the terminol- 
ogy of I .JLR j. see Definition 1 of section 6). This will then allow us to reduce to the study of the 
group P n uHu^^ to that of the group Pg n uHu~^, which has been carried out in (JLRj. 

We identify the quotient space G/P with a flag manifold given by sequences (called n-flags) 
C Vi C ... C Vf_i C V = K", where Vj is a vector subspace of V, of dimension ni + ■ ■ ■ + Uj. 
Studying the double classes of H\G/P, is then equivalent to understand the i/-orbits of the flag 
manifold G/P. This is done in the following theorem. 

Theorem 2.1. The H -orbits of the flag manifold G/P, are characterized by the integers dim{Vi fl 
V"), for l<i<j<t — I, which means that two n-flags D = C Vi C ... C Vt_i C V and 
D' = C V( C ... C V/_i C V are in the same orbit 

under H, if and only if dim{V^ n F/) = dim{V/ n V"^) for I < i < j < t - I. 

Proof. We flrst state the following classical lemma. 
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Lemma 2.1. Let V = K" , and Vp — C V, the F-subspace of vectors of V fixed by a. 
A vector subspace V of V verifies that V' = V'"' , if and only if one can choose a basis of V' 
in Vp, in which case one says that V is defined over F. Any subspace defined over F, has a 
supplementary subspace defined over F. 

Now we prove a second lemma about the filtration of V in terms oiViCiVj' for Vi and Vj in 
the set of subspaces defining a h-Rag. 

Let D be a n-flag, given by the sequence D = C Vi C ... C Vt_i C V. We set Vq = and 
Vt = V. 

For 1 < i < j < t, we denote by Sij a supplementary space of Vi fl V^li + Vi^i fl Vj' in Vi fl Vj'. 
Hi = j, we add the condition that the supplementary space Si^i we choose is defined over F, 
which is possible according to Lemma [2711 

Eventually, for 1 < i < j < t, we denote by Sj i, the space Sf^, which is a supplementary space 

of V, n Vf_, + v,^, n V- in v, nvf . 

Lemma 2.2. With these notations, if {i,j) belongs to {1, . . . ,t}^ , the space Vi-i + Vi Ci V^ is 
equal to the sum 

e . . . © Sx^t) © ... © e . . . © s^-Y,t) © {s^,Y © ... © S'ij). 

In particular, the space Vi is equal to the direct sum 

© ... © ^i,*) © ... © © ... © Si^t)- 

Proof. Let x belong to Vi-i + fl Vj" , then one can write x — Xi^i + yij with Xi-i in 
and yij in Vi n Vf . But then y^j = yt-ij + J/ij-i + s^j with yi-i,j in V^-i n Vj'^ , Vi,j~i in 
Vi n Vj-i' and Sij in Sij. Hence we have x — x[_i + yi.j^i + Sij, with x'i_i — Xi-i + yi-i,j 
belonging to Vi-i. So x belongs to {Vi-i + Vi H V-'_i) + Sij. But by definition of Sij, the 
preceding sum is actually direct, i.e. x belongs to {Vi-i + 1^ fl V-'_i) © Si^j. We thus proved that 
Vi_i + Vin Vj" = {Vi-i + Vi n T/li) © 5"^^, and the proof ends by induction. 

□ 

Getting back to the proof of Theorem 12.11 it is obvious that if two n-flags D and D' are in 
the same i?-orbit, then one must have dim{Vi fl Vj') = dim{V- fl Vj) for 1 < i < j < 
Conversely, suppose that two fi-flags D and D' , satisfy the condition dim{ViriVj') = dim{V-r\V'j) 
for 1 < i < j < t. 

The assumption dim{Vi fl V^") = dim{V/ fl VJ) for I < i < j < t, implies that for any couple 

G {l,...,t}^, Sij and S'^j have the same dimension. For 1 < z < j < t, we choose a 
-fC-linear isomorphism hij between Sij and S'^j. This defines an isomorphism hj^i between Sj,i 
and S'j^, verifying hj^i{v) = {hij(v'')Y for all v in Sj^i. 

Eventually, for each I between 1 and t, as Si^i and S[ ; are defined over F, we choose an isomorphism 
hi^i between Si^i and S'n, verifying that hi^i{v'^) = hi^i[vY for all v G Si^i. 

As the space V is equal to the sum © Sk,i, and V is equal to © S*^;, the 

(fe,i)G{l,....t}^ ' (fc,OG{l,...,t}^ ' 

-fC- linear isomorphism h = © hi^k defines an element of H, sending D to D' , so that D 

{k,i)e{i,...,t}^ 

and D' are in the same i/-orbit. □ 

The proof of the previous theorem has as a consequence the following corollary. 

Corollary 2.1. The quotient H\G/P is a finite set, and its cardinality is equal to the number of 
sequences of positive or null integers {ni,j)i<i<j<t, such that if we let nj,i be equal to nj^i, then 
for i between 1 and t, one has Ui = X)5=i ■ 

Definition 2.1. We call I{n) the set of sequences described in the preceding corollary. 
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Now to such a sequence, we are going to associate an element of G, which will be a represen- 
tative of the corresponding double coset of H\G/P. This will thus achieve the description of the 
set H\G/P. 

First we recall that we denote by V the space K", and that P corresponds to a partition n = 
(rii, . . . ,nt) oi n. We denote by = (ei, . . . , e„) the canonical basis of V, and by the canoni- 
cal n-fiag defined over F, given by C C V2 C V°_i C V, with V° = Vect{ei, . . . , e^ni+...n,)), 
corresponding to the sequence n-ij = if i < j and rii^i = n^. 

Proposition 2.1. (Representatives for H\G/P) Let {nij)i<i<j<t be an element of I{n). We 
denote by V^j the space 

Vect{e(^nx+---+n,-i+n^,i + ---+n,,,-i + l)-, ■ ■ ■ i ^{ni^ hn,-i+Tii,iH hni, j- 1 +ni,j ) ) J 

and we denote by ^ its canonical basis 

{^(niH |-ni-i+ni,iH hni,j-i-|-l) i ■ ■ ■ J ^(niH h"i-l+ni,iH hrii. j-i +";. j ) } ■ 

Hence one has 

V - ® • • • © ® (F2°i © • • • © © • • • © (^M © • • • ® V°t)- 

We denote by u' the element ofG sending V^^ onto itself, and © onto itself for i < j, whose 
restriction to Vl^^ has matrix Inn basis B^^, and whose restriction to jj = Vi^j © Vj^i 

has matrix ^ ^ 

in the basis B?. = U 

The element u' is a representative of the double coset of H\G/P associated with (fM j)i<i<j<t in 
I{n). 

Proof. If Bi = (vi) and B2 — (wi) are two families of vectors of same finite cardinality in V, we 
denote by XBi + /ii?2 the family {Xvi + fiWi) for A and in K. With these notations, the element 
u' sends Vf'j onto Sij = Vect{B^j — j^Bj^), and Vj'^ onto Sj,i = Sfj. Denoting V^^ by Si^i and 
one verifies from our choices that for 1 < i < t, one has Vi = uiVi^) — {Si^i ffi . . . © Si^t) ffi ■ • • © 
(Si,! © ... © Si^t), and that Sij is a supplementary space of Vi fl V^li + Vi-i fl V" in Vi D V^ . 
Hence the fi-fiag D corresponds to the sequence {ni,j)i<i<j<t of I{n). □ 

A reformulation of what precedes is the following. 

Proposition 2.2. (Representatives for P\G / H ) A set of representatives of P\G/H is given by 
the elements u = u'~^ , where the v! are as in Proposition \2.1\ The representative of the class 
associated with the sequence {nij)i<i<j<t in I{n), restricts to V^^^ with matrix /„. . in the basis 
Bf,, and to VP -i with matrix 




in the basis ^ j . 

Definition 2.2. We denote by R{P\G / H) the set of representatives described in Proposition \2.2l 

Each element in this set of representatives has the following property, which is immediate to 
check: 

Proposition 2.3. If the element u of R{P\G/H) corresponds to s = ("■ij )i<i<i<t, we write the 
set {1, . . . , 71} as the ordered disjoint union of intervals U /i,2 U • • • U It,t-i U It,t, with lij of 
cardinality ni,j. Then the element w — uu^"' is the involution o/6„ which fixes the intervals li^i, 
and exchanges the intervals lij and Ij,i for different i and j . 

Moreover u is Pg-admissible, i.e. the Levi subgroup Mg is normalized by w. 
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Remark 2.1. To see that we obtain the same set as in |JLR| . according to Proposition 1.1.3 of 
[C], it is enough to check the inclusion C For a in <I>|^^, two cases occur: either a 

belongs to $+, i.e. Lie{Na) lies in a diagonal block riij x n^j , which is exchanged by w with the 
diagonal block rij^i x nj,i without changing the coefficients inside the block, hence w{a) belongs to 
$+, otherwise Lie{Na) lies in a block riij x n^^fe with j < k, which is sent by w to the sub-block 
nj,i X nk,i of the block nj x n^, hence w{a) belongs to — <i>Jj in this case. 
Another consequence of the preceding discussion is that Mg is actually M n (so this is how 
s would appear in [JLRj ). 

3 Structure of the group P{K) n uGn{F)u~^ and modulus 
characters 

Let u be an element of R{P\G/H), corresponding to a sequence s = ("ij)i<i<j<t in ^(^)- We 
want to analyze the structure of the group P fl uHu"^. If u is P- admissible (which is if s equals 
(ni, . . . , fit)), it is done in |JLRj . However, the group PHuHu'^ is actually equal to PsHuHu"^ , 
we then refer to |JLR) . as u is Pjs-admissible. 

Proposition 3.1. Letu be an element of R{P\G/P), corresponding to a sequence s — {nij)i<i<j<t 
in I{n), the group P n uHu^^ is equal to Pg H uHu^^. 

Proof. Denoting by w the element uu~'^ , the group uHu~^ is the subgroup of G fixed by the 
involution : x ^ 'w~^x"'w, we denote it by G"^^^ , and we more generally denote by S"^^^ the 
fixed points of 9 in S, for any subset S of G. Hence we need to show the equality P<^> = Pf^^, 
which is equivalent to the inclusion P<^> c Ps- We actually show the stronger inclusion PnP™ C 
Ps (here P™ denotes w~^Pw which is equal to 0{P) as a fixes P). As we are dealing with GL{n), 
it is enough to show the inclusion of Lie algebras Lie{P) fl Lie{P)^ C Lie{Ps) as the groups in 
question are the invertible elements of their Lie algebra. But then Lie{P) = Lie{N^) © Lie{Ps) 
and Lie(P) n Lie(P)"' = Lie{N-) n Lie{P)'" © Lie{Ps) n LieiP)"" , because as w belongs to W, 
all of these algebras decompose as a direct sum of subspaces Lie{Na) for some a in <& and maybe 
of the Lie algebra Lie{T). 

We show that Lie{N~) n Lie{P)^ is zero by showing that Lie{N~)^ n Lie{P) is zero, as w is 
an involution. But if a is such that we have Lie{Na) C Lie{N~), then a belongs to $Jj — <i>J. 
As $s = ^AfnM™, we must have w{a) outside but because of Remark 12. 1^ we know that 
w($^_f) C hence w(a) lies in <i>~-$^^. 

□ 

Now Lemma 21 of |JLR| applies, and we deduce: 

Proposition 3.2. Letu be an element of R{P\G / H) , corresponding to a sequence s — ini,j)i<i<j<t 
in I{n), and let w be the involution uu^" of the Weyl group of G. We denote by the in- 
volution X ■w~^x"'w of G. The group P<^> = P n uHu^^ is the semi-direct product of 
M/^> = Ms n uHu-^ and N<^> = iV, n uHu^^ . 
The group Mf^^ is given by the matrices 

"^1,1 

Ai.t 



At,i 

At,t. 
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with Aj^i = in Mn, j{K). 

We will use the following fact later. 

Proposition 3.3. // we denote by P'^ the standard parabolic subgroup of M associated with the 
partition of n corresponding to the sequence s — {ni,j) in I{n), by u the element of R{P\G/H) 
corresponding to s, and by N'^ the unipotent radical of Pi,, then the following inclusion is true: 

N'^ C Nf^>N. 

Proof. Let x be an element of an elementary unipotent subgroup of N'^. We already saw 
the inclusion w{^\j - C <&+ - <^\j, which implies 9{N'^) C N, so 9{x) belongs to N. The 
elements of Na and Nyj{a) commute. If it wasn't the case, then a + w{oi) would be a (positive) 
root. But w is an involution in 6„, writing a as — Cj for i < j, so that w{a) = e^,(i) — e^(j), if 
a + w{a) was a root, one would have w{i) — j, i.e. w would exchange i and j, and w{a) wouldn't 
be positive. 

The consequence of this is that x9{x) is fixed by 9, which implies that x belongs to Nf^-^N. 
Now suppose we know that all y in Ni. of the form Xn ■ ■ - Xi, for Xi in an elementary unipotent 
subgroup of N!,, belong to Nf^-^N. Let z be of the form z = Xn+iy for y of the form described 
just before, and Xn+i in an elementary unipotent subgroup Na of N!,. Then by hypothesis, there 
is y' in TV such that h = yy' belongs to Nf^-'. But 9{xn+i) belongs to N, and as h belongs to 
Nf^^ C P, the element h~^9{xn+i)h is in N. Finally zy'h~^9{xn+i)h — XnJ^i9{xn+i)h belongs 
to Nf^^, and y'h^^9{xn+i)h belongs to N, so z belongs to Nf^-^N. This concludes the proof 
as any element of Ni. is a product of elements in root subgroups of A^^ . □ 

Proposition 3.4. One has the following equality: 

Proof. The group M<^> is the F-points of a reductive group (as it is the fixed points of Ms under 
an involution defined over F). As this is an equality of positive characters, by Lemma 1.10 of 
[KTj . it is enough to check the equality on the (F-points of the) F-split component Zf^^ (the 
maximal F-split torus in the center of Mg) of Mf^^. In our case, the group Zf^^ is given by 
the matrices 

Ai,i/i,i 

Ai,t/i,t 

A2, 1-^1,1 

At,i/t,i 

^t,tlt.,t_ 

with Aij = Xj^i in F*. 

The characters (5p and (5pj verify (5p(a;) — \det{Ad{x)\Lie(N))\K SinA 5 p'^{x) ~ \det{Ad{x)\Lie(N'))\K , 
hence the equality 5p5pi^ ~ 5p^ holds as we have Lie{Ns) = Lie{Ng) © Lie{N). But Zf^^ is 
a subgroup of Ps{F), and one has the relation Sp^{t) — 5p^(p)(i) for t in P„(F), because | \k 
restricts as | ||, to F. So finally we only need to prove the equality Sp (F) = Sp<e> = Sp<e> on 
Zf>. 

We denote by ^a.w{a) the F-vector space {x G Lie{Na) + Lie{N^i^a)) ■ 9{x) — x} of dimen- 
sion |{a,w(a)}|, so that Lie{Nf^^) is the direct sum of the subspaces ^a.w{a) for {ct,w{a)} C 
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Let t be in Zf^^, one has: 

5p<e>{t)= \[ \det{Ad{t)\^^^^J\F ^ n Wii)\F- 

{Q!/u)(Q)}c*+-*t {Qe$+-*t: iu(Q)e*+-*+} 

But we also have 

n Ht)\F^ n i«wif=i. 

The firs equaUty comes from the equaUty w{^s) — The second comes from the fact that 
as t belongs to Zf^^ (so that t™ — t"' — t), we have |a(i)|F| — w{a){t)\p = 1 if a 7^ —w{a), 
otherwise a{t) — ~w{a){t) = a{t)^^ impHes |a(i)|F = 1- 

Multiplying both equalities, we finally get: 

5p<e>{t) = Yl \a{t)\F = ^P,(F)W. 
{aG<S>+-<S>+} 

□ 

4 Distinguished generic representations and Asai L-functions 

We recall that an irreducible representation tt of Gn{K) is called generic if there is a non trivial 
character ip of {K, +), such that the space of linear forms A on V, which verify A(7r(n)w) = ^jj{n)v 
(where by abuse of notation, we denote by ipi'^') the complex number "0(^11.2 + ■ • • + n.n-i,n) for 
n in Nn{K) and v in V ^ is of dimension 1. 

If TT is generic, the previous invariance property holds for any non trivial character ip of K. A 
generic representation is isomorphic, up to unique (modulo scalars) isomorphism to a submodule 
of /nd^"|^j(V'). We denote W^tt, ^) this model of tt on which Gn{K) acts by right translation, 
and call it the Whittaker model of tt. 

In |F4j . the Asai L-function L^{Tr) of a generic representation tt is defined "a la Rankin-Selberg" 
as the gcd of a family of integrals of functions in iy(7r,-0) depending on a complex parameter s, 
for ^ trivial on F. We refer to Sections 3 and 4 of [M3| for a survey of the main properties of the 
Rankin-Selberg type Asai L-function of a generic representation. 

The following theorem due to Zelevinsky (Th. 9.7 of [Zj), classifies the generic representations of 
the group Gn{K) in terms of quasi-square- integrable ones: 

Theorem 4.1. Let n — (ni, . . . ,nt) be a partition of n, and let Ai be a quasi-square-integrable 
representation of GmiK) fori between 1 andt, the representation tt = Ai x • • • x At of the group 
Gn{K) is irreducible if and only if no Aj 's are linked, in which case tt is generic. If (mi, . . . , mf) 
is another partition of n, and if the A^ 's are unlinked segments of length nij for j between 1 and 
t' , then the representation it equals A'j^ x ■ • • x A^, if and only if t — t' , and Ai = A'^i^.^ for a 
permutation s o/{l,...,t}. Eventually, every generic representation of Gn{K) is obtained this 
way. 

Now from Proposition 12 of |F2j, an irreducible distinguished representation tt of the group 
Gn{K) is Galois-autodual, which means that the smooth dual tt^ is isomorphic to tt'^. A conse- 
quence of this fact and of Theorem l4.1l is the following. If tt = Ai x • • • x At is a generic represen- 
tation as in the statement of Theorem 14. II and if it is distinguished, then there exists a reordering 
of the Ai's, and an integer r between 1 and t/2, such that A°^]^ = A,^ for i = 1, 3, .., 2r — 1, and 
Af = A^ for i > 2r. According to Theorem 6 of [K], this means that there exists a reordering 
of the Ai's, and an integer r between 1 and t/2, such that A°^]^ = A,^ for i = 1, 3, .., 2r — 1, and 
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such that Ai is distinguished or 77;<-/j--distinguished for i > 2r. We recall that from Corollary 1.6 
of |A-K-T) . a discrete series representation cannot be distinguished, and ry/f/j^-distinguished at 
the same time. 

Theorem 4.2. Let tt = Ai x ■ • • X At be a generic representation of the group Gn{K) as in 
Theorem \4-l[ it is distinguished if and only if if there is a reordering of the Ai 's, and an integer r 
between 1 and t/2, such that Af^^ = A]' for i = 1, 3, .., 2r — 1, and Ai is distinguished for i > 2r. 

It is a consequence of Proposition 26 of [F3], and of the main result of |M4) that representations 
of the form Ai x • • • x At with A'^_^_■^ = A]' for i = 1, 3, .., 2r - 1, and A^ distinguished for i > 2r, 
are distinguished. 

Before proving the converse fact (i.e. Theorem l4.2p . we recall that from the main result of |M3| . 
this result is known to imply the equality of the Rankin-Selberg type Asai L-function Lp{tt) for a 
generic representation tt of G„ (K) , and of the Asai L-function Lp (p) of the Langlands parameter 
p of TT (see definition 2.4 of |M3j '). Hence the following result is also true. 

Theorem 4.3. Let tt be a generic representation of the group Gn{K), and let p be the represen- 
tation of dimension n of the Weil-Deligne group of K , corresponding to tt through Langlands 
correspondence. Then we have the following equality of L-functions: 

Lf(^,s)=if(p,s). 

From the discussion before and after Theorem I4.2[ the proof is then reduced to showing the 
following fact. 

Theorem 4.4. Let tt = Ai x • • ■ x At be Galois autodual generic representation of the group 
Gn{K), if it is distinguished, then there is a reordering of the Ai's, and an integer r between 1 
and t/2, such that Af^_]^ = A/ for i = 1, 3, .., 2r — 1, and Ai is distinguished for i > 2r. 

Proof. Let n = (ni, . . . , Tit) be the partition of n corresponding to tt. We suppose that the A^'s 
are ordered by length. Moreover as tt is Galois autodual, we suppose that inside a subsequence of 
same length representations, the first to occur are the non Galois autodual, and that at the first 
occurrence of such a Ai^ , its successors are alternatively isomorphic to A^"' and A.^^ , until no 
representation among the A^'s is isomorphic to A^^ (hence such a sub subsequence begins with a 
Ai isomorphic to A^^, and ends with a A^ isomorphic to A^°'). 

Let A be the representation Ai (g) • ■ • (g) At of P, from Lemma 4 of |F4) . the ff-module tt 
has a factor series with factors the representations ind^^ip^^j^KSp^ A)") (with (5p^A)"(a;) = 

Sp^ A{uxu~^)) when u describes R{P\G/H). Hence if tt is distinguished, one of these represen- 
tations admits a nonzero i7-invariant linear form on its space. This implies that there is u in 
R{P\G / H) such that the representation indp^^^^_i (J^/^A) admits a nonzero uHu~^ -invariant 
linear form on its space. Then Frobenius reciprocity law says that HorriuHu-^ ('^''^'^p^uHu-^ i^p^^)^ 1) 

is isomorphic as a vector space, to Hompr^^Hu-^ ((^p ^ A, Spr^uHu-^ ) = HomprtuHu-^ {^p^ /^PnuHu-^'^, 1)- 
Hence there is on the space Va of A a linear nonzero form L, such that for every p in PO uHu~^ 

^1/2 

and for every v in Va, one has L{x{p)A{p)v) = L(v), where x{p) = i — ~ (p)- both 

and <^pntiffM-i ^ire trivial on O uHu~^, so is x- Now, if s is the element of /(n) 
corresponding to u, let n' belong to N!,, from Proposition 13.31 we can write n' as a prod- 
uct nsUo, with rig in TVs H uHu~^, and ng in N. As N is included in Ker{A), one has 
L{A{n'){v)) = L{A{nsno){v)) = L{A{ns){v)) = L{xins)A{ns)v) = L{v). Hence L is actu- 
ally a nonzero linear form on the Jacquet module of Va associated with A'^^. But we also 
know that L(x(ws)A(tos)w) — L{v) for m in Ms{F), which reads according to Lemma 13.41 

— 1/2 

L{6p, {ms)A{ms)v) — L{v). 

This says that the linear form L is Ms(P)-distinguished on the normalized Jacquet module 
?'Ms,m(A) (as Ms is also the standard Levi subgroup associated with A^^)- 

The following lemma will conclude the proof of Theorem 14.41 
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Lemma 4.1. Let Ai, . . . , At he unlinked segments of respectively Gni{K), . . . , Gnt{K), such 
that the set {Ai, . . . , Ai}, is stable under the involution A ^ IS^" , call n the integer ni + ■ ■ ■ + nt, 
and n the sequence (ni, . . . , nt). Suppose moreover that the A; 's are ordered by length, and that 
inside a subsequence of same length representations, the first to occur are the non Galois autodual, 
and that at the first occurrence of such a segment A^^ , its successors are alternatively isomorphic 
to Ar and A^Q, until no segment among the A^ 's is isomorphic to Ajj,. Then if there is s — 
{'>^i,j)i<i<j<t inl{n), such that rM,,M{®iA.i) is Ms{F)-distinguished, there exists a reordering of 
the Ai 's, and an integer r between 1 and t/2, such that Af_^_^ = A^ for z = 1, 3, .., 2r — 1, and Ai 
is distinguished for i > 2r. 

Proof of the Lemma. We do this by induction on t. It is clear for t = 1. 
Now suppose the result to be true for any t' < t. 

If there is s in I{n), such that rM^,M{A) is Ms(i^)-distinguished, in particular rM^,M{A) is 
nonzero, which from Proposition 11.11 implies that one can write each A^ under the the form 
[Ai^i, . . . , Ai^t], with Aij a subsegment of A^ of length Uij. With these notations, tm^.mCA) 
is equal to the tensor product Ai^i ® ■ ■ ■ ® At^t- Hence the fact that rM^,M(A) is distinguished 
by the group Ms{F) is equivalent to the fact that Aij is Gm ; (F)-distinguished if riij ^ 0, and 
Aj- J = A,^J if i < j and ^ 0. 

Let io be the smallest i, such that Ai^i (or equivalently rti,i) is nonzero. 

1. io = 1: the representation Ai^i is distinguished, hence Galois autodual. If Ai = [Ai^i, . . . , Ai^t] 
was not equal to Ai^i, then one would have A^°" = [A^^ , . . . , A14] ^ Ai. But the segment 
A^'^ would also occur, and Ai and A'^"' would be linked, which is absurd. Hence Ai = Ai 1 

is distinguished, and rti_i = if i > 1. We conclude by applying our induction hypothesis 
to the family A2, . . . , Af , the integer n — ni with partition (^2, . . . , nt), and sub-partition 
s' = i>2,j> 2). 

2. io > 1: one has A^^^i — A^."'^. As the representation Ai„ is either Galois autodual, or 
coupled with A^"', the representation A^"^ = [A^'^j, . . . , Ai^i,,] occurs. But because the 
representation Ai has the smallest length among the A^'s, the segments Ai and A^*^ would 
be linked unless Ai — Ai ^q, which thus must be the case. In particular one has ni^i = 
for i ^ io- 

Two cases occur. 

a) Ai = Ai^ig is Galois autodual: if A^^ wasn't equal to A^^, 1, then the two occurring 

segments A^^ = [A^^^i, . . . , Aig,t] and A/^'^ = [A/^'^j, . . . , Aig,i] would be linked, and 
that is not the case. Hence we have A^^ = A^^^ = Ai, and rii^j = for j 7^ 1. We con- 
clude by applying our induction hypothesis to the family A2, . . . , A^g-i, Ai^+i, . . . , At, 
the integer n — ni— Uig with partition (ri2, • ■ • , ni„+i, . . . , nt), and sub partition 

s' = (riijl i ^ io, j ^ 1). 

b) Ai = A is not Galois autodual: in this case A2 is A^"^ because of our ordering. 

Let jo be the smallest j, such that A2J (or equivalently n2j") is nonzero. If jo = 2, as 
in the case io = 1, one has A2 — A2,2, and we conclude by applying our induction hy- 
pothesis to the family Ai, A3, . . . , At, the integer n — n2 with partition (m, n3, . . . , nt), 
and sub partition s' — {nij \ i^2,j^ 2). 

If jo ^ 2, then A2 must be equal to A2J0. It is indeed clear for jo > 2, otherwise A2 
and Ajj^ would be linked, and in the case jo = 1, then one has iq = 2, and A2 is equal 
to [A2,i, . . . , A2,f] but also to A^'^ = [..., A2,i], so that A2 is A2,i- 
This implies ^2^ = for j ^ jo- Thus we have A2 = A2j„ = A^J^ = A^'^. But the 
two occurring segments A/^'^ = [A/^-^j, . . . , Ai] and Aj„ = [Aj„,2 = Ai, . . . , A^^^t] will 
be linked unless either A^^ = A^*^ , in which case we conclude by applying our induction 
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hypothesis to the family A2, . . . , AjQ_i, Aju+i, . . . , At, the integer n — m — rijo with 
partition (n2, . . . , ni^-i, ni(,+i, . . . , n*), and sub-partition s' = {nij\ i ^ io, j ^ 1), 
or Ajg = Ai, in which case wo conclude hy applying our induction hypothesis to the 
family (Aj| j ^ 2 and jo), the integer n — n2 — nj^ with partition {nj\ j ^ 2 and jo), 
and sub partition s' = {nij\ i ^ 2,j ^ jo). 

□ 
□ 
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